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After task completion, students were unexpectedly tested on meaning recall of 
target words. Researchers investigated possible connections between retention 
and lookup behavior (type of information selected and number of lookups per 
word). Overall, different students had different lookup preferences. The use 
of multiple dictionary information reinforced retention. Results highlight 
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ABSTRACT 

The study investigates a relationship between what is looked up about new words 
when different kinds of information are available and how well these words are 
remembered. The dictionary information has been incorporated into a CALL 
programme which is comprised of a text, highlighted low-frequency words and 
access to different lexical information about these words (explanation in English, 
translation into LI, sound, root, and "extra" information). 

The subjects were English as a foreign language (EFL) university learners in Hong 
Kong and Israel. The target words examined for incidental learning were 12 low 
frequency words. Pre-tests showed that they were unfamiliar to niost subjects. The 
subjects were asked to read the text on the screen and understand it so that they 
could take a comprehension test after reading it. Unknown words could be looked 
up in the CALL dictionary built into the programme. During the task, log files 
registered every selection of dictionary information. After task completion, subjects 
were unexpectedly tested on meaning recall of the target words. 

Recall data were analysed (ANOVAs, repeated measures, and correlations) to 
establish possible connection between retention and lookup behaviour (type of 
information selected and number of lookups for each word). Results suggest that 
different people have different lookup preferences and that the use of multiple 
dictionary information seems to reinforce retention. The teaclung implication is, 
therefore, to provide a variety of lookup options catering to different lookup 
preferences in paper or CALL dictionaries when assigning tasks that involve 
reading comprehension and understanding of unfamiliar words. 

BACKGROUND 

Attention to the form of input has occupied much of recent SLA research (e.g., Fotos, 1993; 
Robinson, 1995; Schmidt, 1990, 1993). There is growing research evidence that L2 learning, 
particularly adult L2 learning, is impossible without attention to input in the sense of noticing it. 
Though attended learning is discussed in literature in relation to syntax, incidental vocabulary 
learning is no exception to the attention requirement. This may sound paradoxical at first, since 
incidental learning is sometimes mistakenly assumed to be unattended learning. This is not the 
position taken in this paper. Incidental vocabulary is learrit as a by-product of another activity, such 
as reading or communication, without the learner's conscious decision, or intention, to leam the 
words. For example, during a reading activity, words are looked up in a dictionary in order to 
understand the text and to perform a comprehension task. Subsequently, some of these words are 
remembered even though the main task was not a vocabulary task, nor was it the reader s intention 
to leam the words in the text. Learning was thus incidental, that is, unintentional and as a by- 
product of another activity. However, it was not unattended. It is indeed highly debatable whether 
words which arc not noticed in the input can be learnt. 




3 



1 



It is also questionable whether noticing alone will result in acquisition. While this is not 
impossible, in most cases additional elaboration strategies will be necessary on the part of the 
learner before a memory trace for the noticed word is created. In the case of reading, these 
strategies include attempts to infer the meaning of a word from context, consulting a dictionary and 
selecting the meaning that best fits the context, and relating the form and the meaning of the word 
to other words that the learner knows. Cognitive psychologists and language acquisition scholars 
working within the framework of cognitive psychology believe that retention of information is 
determined by the way in which this information is processed (Craik & Lockhart, 1972; Ellis, 

1994; Haastrup, 1989; Mondria & Wit-de-Boer, 1991; Schouten van Parreren, 1989; Watanabe, 
1997). "The more a learner pays attention to a word's morphophonological, orthographic, 
prosodic, semantic and pragmatic features and to intraword and interword relations, the more likely 
it is that the new lexical information will be retained" (Hulstijn, forthcoming). Such close attention 
to word features, which is often associated with performing a vocabul^ task, has also been 
referred to as deep processing, elaboration, cognitive effort-terms which defy a simple definition. 
Retention of words attended to will happen regardless of whether vocabulary learning is intentional 
or incidental. Hence, effective incident^ vocabulary learning is a conscious learmng process. Yet 
deep processing during the first encounter will not in all likelihood induce long term retention. 
There is evidence to show that repeated exposures to the new word in language input reinforce 
learning though it is unclear how many repetitions are necessary for this. Sailing (1959) suggests 
that the number is five, Kachroo (1962) and Crothes and Suppes (1967), suggest it is seven, and 
Saragi, Nation and Meister (1978), sixteen (surveyed in Nation, 1990). I 

If learning is dependent upon attention and quality of information processing, then effective 
teaching should include tasks which direct the learner's attentiori to the words targeted for 
instruction and require elaboration of the words. One such task is the use of dictionanes to Iwk up 
the target words in the course of a reading assignment. It was found that the use of dictionanes or 
glosses can indeed contribute to small increments in vocabulary learning (Chun & Plass, 1996, 
1997; Hulstijn, Hollander & Greidanus, 1996; Knight, 1994; Luppesku & Day, 1993; Lyman- 
Hager & Davis, 1996; Lyman-Hager, Davis, Burnett, & Chennault, 1993; Mondna, 1993). 

The advent of electronic dictionaries inspired research of how these dictionaries are used, and their 
usefulness as on-line helping tools and as contributors to incidental vocabidary learmng. Leffa 
(1992) investigated the efficiency of an electronic dictionary and a conventional dictionary in a 
translation task and found that the computer dictionary enabled the students to understand 38% 
more of the passage, using 50% less time" (p. 63). Knight's ( 1994) study compared the effect of 
CALL dictionary lookup with guessing words from context and found that students who used a 
dictionary learnt more words and achieved higher reading comprehension scores. Research into the 
influence of task and learner variables showed that words wMch were deemed relevant for the task 
were looked up more frequently than those which were considered irrelevant. In addition, words 
whose meanings could be guessed from context were less likely to be looked up than those whose 
meanings could not easily be inferred (Hulstijn, 1993). 

The above research, revealing as it may be, has two major limitations; uncertainty about wWch 
words were looked up and limited dictionary information provided for learners. Some studies 
report that electronic or paper dictionaries were available to the class. This, in itself, however, does 
not necessarily mean that leaimers looked up the words which the researcher assumed would be 
looked up. If a study does not provide log files which record what learners are doing during the 
reading task, there is no evidence that they are indeed looking up unknown words, rather than 
guessing or ignoring them. Nor do we have the information about the number of times they return 
to a specific word during the reading task. Lack of certainty about the use of dictionaries is even 
more evident in the case of paper dictionaries. Other studies, which overcome this limitation by 
tracking learners' look up behaviour in log files, research only one type of dictionary selected by 
the researcher— monolingual or bilingual. The learner is thus denied the opportunity to select the 
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dictionary s/he would feel most comfortable with in real life. And yet different people, when given 
the choice, consult different types of dictionary information (Laufer & Kimmel, 1997). Some 
prefer translations, some explanation in L2, others a mixture, and some access different 
information for different words. Moreover, the studies cannot reveal whether the learners read the 
entire entry or a part of it, and which part (assuming the dictionary used was monolingual). 

Because of these shortcomings we may be missing important information about what words are 
looked up, how many times they are looked up and whether the dictionary information that has 
been provided would be the kind of information the learner would have selected in real life. 

More rigorous research of electronic dictionaries would require research tasks and a computer 
programme that would overcome the above limitations. As for tasks, the researcher has to make 
sure the tasks cannot be carried out without the knowledge of the words targeted for investigation. 
Relevance of the words to the task will increase the chance of dictionary consultation (Hulstijn, 
1993). Second, as we cannot be sure whether the learner will look up the target words even when 
they are relevant to the task, log files should track learners' lookup behaviour to show whether 
indeed they looked up the target words or additional words, and how many times they select^ 
each word. Third, the programme should provide options for selecting different types of dictionary 
information for each word. If, for example, the learner is interested in a quick L2-L1 translation, 
the option should be available. If, on the other hand, s/he is interested in examples of usage in 
grammatical information, or in a definition, each type of information should be accessible via 
another lookup option. Lx)g files would record which of these options were selected for which 
words. 

Studies that satisfy the above requirements— eliciting target word lookup, providing access to any 
and all types of dictionary information, tracking students' lookup behaviour— can provide insight 
into the effect of dictionary use on incidental vocabulary learning. Specifically, we can investigate 
the relationship between retention of looked up words and the type of dictionary information 
selected, and between retention and the number of times a word was looked up. In addition, 
providing different dictionary information and tracking people's selection of iiiformation is a more 
rigorous method of investigating dictionary preferences than questionnaires which have often been 
used in dictionary use studies. Several recent studies which have included multimedia annotations 
have been able to shed more light on learner lookup behaviour (Aust, Kelley, & Roby, 1993; Chun 
& Plass, 1996; Davis & Lyman-Hager, 1997; Lomicka, 1998; Lyman-Hager et al., 1993; Roby, 
1991, 1999) 

Taking advantage of the tracking capabilities of computer mediated language learang, Roby 
(1991) cited in Roby (1999) examined the reading comprehension level of American tertiary 
students of Spanish. Computer and paper modes of presentation were used to compare the effect of 
reading with and without glosses. Roby focused on reading time, number of lookups and 
comprehension and found that there was no significant difference in comprehension. However, 
subjects who had access to a gloss read the passage in significantly less time than those in the 
dictionary alone treatments, and those who used an electronic dictionary looked up sigmficantly 
more words than those who used a papjer dictionai 7 . In a similar study comparing an online 
dictionary aid and conventional paper dictionary, Aust et al. ( 1993) noted that those using the 
electronic reference had more than twice as many dictionary lookups as those in conventional 
mode, but again there was no significant difference in comprehension. 

Incidental vocabulary learning through an L2 reading comprehension task was the focus of a study 
by Chun and Plass ( 1996). Three types of multimedia annotations were tested: text, text plus 
image, and text plus video. One hundred sixty American students of German were introduced to 
Cyberbuch, a hypermedia application for reading German texts containing a variety of annotations 
such as those just mentioned. Their results showed incidental learning of 25% accuracy in 
production tests and 77% in recognition tests with minimal loss between immediate and unexpected 
delayed recall. In their 1998 study, Plass, Chun, Mayer and Leutner presented another group of 



English speaking tertiary students of German with a 762 word text annotated with verbal (LI 
translation) and/or visual (picture or video clip) annotations. A higher level of lexical recall was 
found with the students who had selected both visual and verbal annotations and those who were 
able to select their preferred mode of annotation showed better comprehension. 

While the previous studies focused on reading comprehension and vocabulary acquisition, 
Lomicka (1998) analysed the think aloud protocols of 12 native English speakers during a 
computerised reading task in French to find out whether glossing aids comprehension. Subjects, 
who were asked to read a text in French were divided into three groups: A, text only (no glosses); 
B, text plus traditional glosses (LI translation and L2 definitions); and C, access to B's glosses 
plus pronunciation, images, references, and questions. A tracker recorded the amount and type of 
glosses and the length of time that each was consulted. The data suggested that computerised 
reading with all glosses "may promote a deeper level of text comprehension," however, the main 
obstacle to comprehension was vocabulary. 

HOW DICTIONARY LOOKUP PATTERNS CAN BE INVESTIGATED: WORDS 
IN YOUR EAR 

Before reporting our particular study, we will describe the computer programme called Words in 
Your Ear, which attempts to meet the research requirements outlined in the background section.2 

The programme consists of four parts; (a) a pre-test of the words targeted for investigation, (b) a 
text where these words appear highlighted, (c) dictionary information for each word in the form of 
five options (meaning in English, translation into LI, word pronunciation, root, and "extra" 
information), and (d) log files where every mouse click selecting from these options is recorded. 

The pre test displays the words on screen asking the student to write the meaning next to them if 
they are familiar to him/her (see Figure l) . 

Figure 1 Pre test screen 

The next screen displays the text "Meeting Mania" in which the 12 target words are highlighted in 
red (as opposed to the black script of the text). The right side of the screen provides the lookup 
options for each word, as in Figure 2 . 



Fiyure2 Text of Meeting Mania with lookup options 

Unfamiliar words can be looked up by clicking on them with the mouse and choosing any of 
several options on the right side of the screen. Learners can return to any word any time while 
reading the text for further information. The following information about each word is provided 
under separate options: 

1 . Hear word (pronunciation in the form of a digitized voice recording); 

2. English meaning (the definition in English and contextualised examples taken from the 
Longman Active Study Dictionary of English [Summers, 1984]); 

3. LI meaning (in Chinese or in Hebrew using translations taken from Segal and Dagut's 
English-Hebrew Dictionary [1986] and Longman’s Active Study English-Chinese 
Dktionc^ [Li, 1995]); 

4. "extra" information (other forms of the word, phonemic transcription, details of levels of 
formality, prepositions which follow the item, related meanings and other semantic and 
syntactic details); and 

5. root (taken from Chambers Twentieth Century Dictionary [Geddie, 1968]) 
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Fiolire 3 shows the screen print with the word burgeoning selected and its English meaning, while 
Figure 4 shows the Screen print with the word insidious selected and its Hebrew translation. 

Figure 3 Example of the English meaning of burgeoning 

The student can also opt to hear the text being read aloud. The line along the bottom of the screen is 
a timer which prompts the students when the maximum time of ten minutes is exhausted. 

Figure 4 Example of the Hebrew translation of insidious 

Whenever a student selects information by clicking on it, the log registers the click in a log file. 
Figure 5 shows an example of a student log at the end of the task. 

Figure 5 Example of a student log 

The right part of the file has recorded the learners' responses to the words during the pre-test and 
the left part reveals the lookup behaviour. A zero means that the information was not selected, "1" 
or "2" means that the relevant option was clicked on once or twice and "times selected" indicates 
the number of times the word was returned to: the student may have selected several options when 
first reading the text, then made a further selection during a subsequent reading. 

The results screen (see Figure 5 1 shows that in the pre-test the learner has perceived the meaning of 
insidious to be the "adjective from inside" and noted assert as meaning "claim." She has selected 
the English meanings of 8 of the 12 words, checked the Chinese translation of 5 and listened to the 
pronunciation of 6 words. She has chosen to look up the extra information on 2 words, pervasive 
2 nd congregate, and has checked the roots of 4 words. She also opted to hear the text being read 
aloud. 

Using the above programme as our research tool, we set out to investigate the relationship between 
L2 learners' dictionary lookup patterns and their retention of the looked up words. 

THE STUDY 

Research Questions 

Our specific research questions were as follows: 

1 . What percentage of words are remembered after being looked up in an electronic dictionary 
during.a reading task? 

2. Are different lookup preferences associated with different levels of retention? 

3 . Is there a relationship between the number of lookups and retention? 

Subjects 

Initially 97 subjects participated in the study, but only 72 were left for data analysis, as will be 
explained in the section on pre-test. Of the 72, 32 subjects were EFL students from the University 
of Haifa, Israel, and 40 were first year ESL students from the University of Hong Kong. The 
Israeli students were non English majors taking a course in English for Academic Purposes. They 
had had eight years of English in high school prior to their university studies. Their score on the 
English section of the psychometric university entrance exam was 1-1.5 standard deviations {SD) 
above the mean. Since this section tested reading comprehension only, the students' level is 1-1.5 
SD above the mean on the reading section of TOEFL (National Institute for Testing and 
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Evaluation, personal communication, December 1998 ). The Hong Kong students were from the 
Social Sciences and Arts Faculties. Two students were English majors, however, their English 
grades were not above the group average and their performance was no better or worse than their 
peers. The mean proficiency level of the Hong Kong students was about 570 on TOEFL. All were 
taking English for Academic Purposes and all had had at least seven years of English in secondary 
school in Hong Kong. Even if the two groups were not equivalent on all language skills, they 
could read the text without any difficulty except for the target words. Furthermore, the focus of the 
study was within subject differences in retention as a function of lookup patterns. All subjects 
reported that they were already familiar with a computer environment and knew how to use a 
mouse. 

MATERIALS AND PROCEDURE 
Materials 

The text in the programme is a short extract (120 words) from an academic text of fairly general 
interest. Meeting Mania (Bergman, 1994). Before the experiment, it was piloted with 30 students 
who were not involved in the present study and who were asked to highlight all words of whose 
meanings they were unsure. Twelve words were most frequently marked as being unfamiliar; 

_ assert _ burgeoning _ con^egate _ endeavour 

_ insidious _ malpractice _ mania _ peiwasive 

profusion _ rampant _ scrutiny _ ubiquitous 



These words were therefore selected as the target words to be investigated in the experiment on 
incidental learning. Even though the templates of the programme allow for linking any number of 
words to glosses, in this particular text only the above twelve words were glossed and could 
therefore be looked up. Other words in the text were of high frequency and did not present any 
problems in the pilot. 

Procedure 

The experimental procedure consisted of three stages: pretest, tutorial, and vocabulary retention 
test. 

Pretest. The students logged in and the first screen was displayed with the 12 target words. They 
were asked, on the screen, if they knew the meanings of any of these words (see Figure.i). This 
pretest allowed us to find out whether or not some of the target words were familiar. Words which 
were reported as known and those for which incorrect meanings were given were dso noted. 
Students who were familiar with more than one of the target words were later eliminated from the 
sample. Those who knew one word were later not credited with "learning" that word. So the final 
number of subjects whose data were analysed was 72. 

Tutorial. Having completed the pretest stage, the learners received the second screen which 
displayed the text Meeting Mania with the 12 highlighted target words (see Figure 2). Subjects were 
instructed to read the text and understand it for comprehension questions which would be given at 
a later stage. They were also told that, in the course of reading, they could look up infonnation 
about the highlighted words by clicking on them with the mouse and then choose the options that 
would best clarify the meaning of the word in the text. Students were encouraged to do so since the 
words were relevant to text comprehension. They were not told to learn the words, nor were they 
notified of a vocabulary test which would follow. The fact that the words were highlighted on the 
screen made. them salient to the learner, similar to a marginal gloss in a text. However, enhanced 
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input in itself does not result in a decision to learn it. Since the task was specified as text 
comprehension and not vocabulary learning, we believe that retention of the target words was 
indeed incidental.3 



As mentioned before, the log recorded every mouse click and showed, in the results screen, which 
words were known in the pre-test (if any), which words were selected in the tutorial, which 
dictionary information was looked up, and the number of times each word was selected (see Figure 
5 above). 

The students were warned that the tutorial session would last no more than 10 minutes and that the 
text would disappear from the screen after this. They were asked to notify the researcher or the 
assistant if they finished reading the text earlier. In such cases, they moved away from the monitor 
to zinother part of the room. Most Israeli students spent between 5 and 6 minutes on the text; most 
of the HK students spent close to the 10 minutes allocated. 

Retention Test, On completion of the reading task, subjects were given an unexpected 
vocabulary post-test (on paper) in which the twelve target items were listed. The subjects were 
asked to write the meaning of the words in LI or L2. When they handed in the sheets, they were 
given a comprehension exercise with 6 questions on the text, as announced before the tutorial 
session. 

Data Analysis 

For each subject, the following data were collected from the pretest stage: the number of already 
familiar target words (i.e., with correct explanations provided) and the number of words 
mistakenly perceived as Imown (i.e., with incorrect explanations). As mentioned before, only 
subjects who did not know the target words were selected for tutorial and post-test analyses. The 
log files showed that all the target words were looked up by all subjects. From the post-test, we 
calculated the number of correct responses, that is, the number of words each student remembered 
after the reading task. From the tutorial section and the post-test, we collected data on the lookup 
options and the relation between each option and word retention. In other words, for each lookup 
option we checked how many of the looked up words were later retained. Specifically, we 
calculated how many words were looked up in LI only and of those, how many were remembered 
on the post-test; how many were looked up in English only and of those, how many were 
remembered on the post-test; how many were looked up in both languages and of Aose, how 
many were remembered; and finally, how many were looked up for meaning (in any or both 
languages) and also for additional information (sound, root, extra information) and of those, how 
many were remembered on the post-test. We also noted the total number of lookups for each 
student, including repeated selections of words. 

Students were categorised by their preferred lookup behaviour as "lookup types": those who 
predominantly (in 75% of cases) selected Chinese or Hebrew translations were categorised as LI 
type; those who preferred the English meanings were L2; those who looked up LI and L2 in ^ual 
proportions were LHL2\ and those who selected word meaning (in LI, and/or L2) together with 
additional information were grouped as other. 

In sum, for each of the 72 students, we obtained the following information: the number of new 
words retained after the tutorial, the number of words selected in each lookup option, the number 
of words remembered in each lookup option, the total number of times words were looked up, and 
classification of the students by preferred lookup pattern. 
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The scoring procedure was straightforward. A correct answer received one point, an incorrect 
answer zero points. An answer on the post-test was considered correct if the learner provided the 
meaning given in the CALL programme. When the subjects opted for explanations in English 
rather than LI translations, a correct answer did not necessarily require providing the exact words 
used in the glosses as long as it was semantically accurate. For example, the meaning given for 
profusion might be "large quantity," "big amount," or "a lot." If the answer w^ semantically 
accurate but contained a minor spelling mistake which did not distort its meaning, it was 
considered correct. We expected and received only meanings provided in the CALL programme. 
(The target words were not homonyms.) 

Results 

The first research question was. Are words remembered after being looked up in an electronic 
dictionary during a reading task and if so, how many? 

Table 1 presents the means of retained words. IL stands for the Israeli group of subjects and HK 
stands for the Hong Kong group. 



Table 1 . Mean of total learning scores on post-test (out of 12) 





Mean 


SD 


P 


Min 


Max 


IL 


4.00 


2.50 


33.3% 


1 


10 


HK 


7.45 


2.70 


62% 


2 


12 



Table 1 indicates that the answer to our first question is affirmative. The Hebrew LI subjects 
recalled an average of 4 out of the 12 target words while the Chinese LI subjects recalled an 
average of 7. The maximum recall of the Hebrew LI subjects was 10, or 83%, while the maximum 
number of items recalled by one Chinese LI subject was 12 words, that is, 100%. 

The second research question was: Are different lookup preferences associated with different rates 
of retention ? 

This question was answered in two ways. As mentioned eariier, we collected information on the 
percentage of words correctly retained in each of the four lookup options. TablcjZa shows how 
many words were looked up in each option, and how many words were retained in each option. 
The numbers of words looked up and retained are given in raw scores. The raw scores of the 
Hong Kong students' lookups are out of 480 since the maximum number of words that could be 
looked up in each option in the entire group is 480 (12 words x 40 subjects). The raw scores of the 
Israeli students' lookups are out of 384 since the maximum number of words that could be looked 
up in each option is 384 (12 words x 32 subjects). The number of words retained in each option 
was also converted into percentage (of the number of words looked up in the option). 



Table 2. Words looked up and retained in each option ( 12 words total) 





LI 


L2 


L1+L2 


Ll/L2+other info 




Looked 

Up 


Retained 

% 


Looked 

Up 


Retained 

% 


Looked 

Up 


Retained 

% 


Looked 

Up 


Retained 

% 


IL 

(n=32) 


171/65 


38%' 


70/21 


30% 


53/2 


45% 


90/26 


29% 


HK 

(n=40) 


38/16 


42% 


120/95 


79% 


135/90 


67% 


157/ 97 


62% 



I ANQVA 



SS 



MS 



dL 



F-ratio 
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IL between groups 


1877.5 


625.8 


3 


0.51 


.067 


IL within groups 


66089 


1223.8 


54 






HK between groups 
HK within groups 


6230.4 

45908 


2076.8 

573.84 


3 

80 


3.6191 


.0166 



Analysis of Variance comparing mean retention scores in the 4 lookup conditions showed that there 
was no significant difference between the lookup options among the Israeli learners 
(F(3,54)=0.51, p>0.05). In the Chinese group, the difference between the retention scores was 
significant (F(3,80) =3.6, fx.05). A Tukey Kramer multiple comparisons test shows a significant 
difference between LI and the three other groups (in Table 2a) . 



Table_2a,Tukey Kramer test of differences across lookup options. 



Lookup option 


LI 

Mean= 42% 


L2 

Mean= 79% 


L1+L2 
Mean= 67% 


Ll/L2+other info 
Mean= 62% 


LI 




4.50* 


4.10* 


4.10* 


L2 






0.60 


0.60 


L1+L2 








0.02 



* p<0.05 



Table 2 shows that the most frequent look up strategy of the Israeli learners was LI tr^slation. Yet 
their highest retention score (45%) was associated with selecting both LI and L2 during the 
tutorial. The Chinese learners, unlike their Israeli peers, used LI translations least frequently. Even 
though the mean retention score in LI condition was 42% (which is higher than the Israeli 38% ), 
this was the lowest retention score of the Chinese learners. The most frequent lookup procedure 
which resulted in correct retention of a word was selecting the English meamng. The other two, 
which involved selecting English with LI, or one of the languages plus additional information, 
yielded lower scores, but not significantly different from the L2-oriented lookups. 

The second way of checking the relationship between lookup options and retention was by 
classifying learners by their preferred lookup patterns. Table 3 shows the distribution of different 
look up types of students in numbers and in percentages out of the total number of students in each 
country. 



Table 3. Distribution of lookup types of learners 





LI 1 


type 


L2 type 


L1/L2 


Other 


IL(« = 32) 


23 


72% 


2 


6% 


5 


16% 


2 


6% 


HK (/; = 40) 


5 


12.5% 


13 


32.5% 


7 


17% 


15 


38% 



Like Table 2 . Table 3 shows that the Israeli and the Chinese students exhibit a very different 
dictionary behaviour. The predominant lookup type among the Hebrew LI subjects was LI (72%) 
and few alternated between LI and L2 depending on the looked up word. (16%). Even fewer (6%) 
of the Hebrew LI subjects relied solely on the English meanings of the target items or opted to 
look at the additional information provided by the programme. The Chinese LI subjects, on the 
other hand, tended to select more information about the target items and were predominantly of the 
other lookup type (38%). The next largest grouping (32.5%) selected the English meanings of the 
words, and fewer (17%) chose to alternate between LI and L2. Only 12.5% chose Chinese 
lookups. Table 4 shows the mean number of words and percentage (out of the twelve target words) 
was retained by different "lookup types" of the learner. 



Table 4. Word retention mean scores of different "lookup types" 



LI type 


L2 type 


L1/L2 


Other 


Score % 


Score % 


Score 


% 


Score 1 % 
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IL 


4.0 


33% 


3.0 


25% 


5.0 


42% 


2.5 


21% 


HK 


4.4 


37% 


8.3 


69% 


7.2 


60% 


8.0 


66% 



ANOVA 


SS 


MS 


-JL 


F-ratio 


P 


IL between groups 


11.5 


3.8 


3 


0.59 


0.63 


Within groups 


182.5 


6.5 


28 






HK between groups 


59.340 


19.7800 


3 


3.30 


0.03 


Within groups 


218.560 


6.0711 


36 







Analysis ol' Variance reveals no significant difference among the lookup types in the Israeli group 
(F(3,28)=0.59, p=0.63>0.05) and a significant difference in the Hong Kong group (F(3,36)=3.3, 
p=Om< 0.05). A Tukey Kramer Multiple comparisons test (Table 4a) shows significant differences 
b»etween LI and L2 (p<.05) and between LI and other (jx.OS). 



Table 4a Tukey Kramer test of differences across lookup types. 



Lookup option 


LI 

Mean= 37% 


L2 

Mean= 69% 


L1+L2 
Mean= 60% 


Ll/L24other info 
Mean= 66% 


LI 




4 2* 


2.6 


3.8* 


L2 






1.4 


0.6 


L1+L2 








0.9 



* p<0.05 



Tables 2 and 4 point to similar results in retention as a function of lookup patterns. Even though 
there is no significant difference in retention among the Israeli groups, the highest retention scores 
(45% in Table 2 and 42% in Table 4) are associated with consulting both languages for the new 
words. The next best result seems to be achieved when words are look^ up in the LI. The 
category of Hong Kong learners which attained the highest mean retention score of 69% was the 
L2 lookup type. The next highest scoring group with a mean of 66% was the other type, followed 
by those who alternated between LI and L2 (60%). The group which retained the lowest mean 
number of target words (37%) was the LI type who selected only the Chinese meanings. 

The third research question was; Is there a relationship between the number of lookups and 
retention? 

Table 5 presents the means of number of times selected, that is, the mean number of "clicks" on the 
target words. Table 6 shows the correlations which were calculated by relating each student’s 
number of selections (clicks) and his/her total retention score. 



Table 5 Mean number of selections 





Times selected 




Mean 


SD 


IL(n = 32) 


18.6 


9.6 


HK (n = 40) 


30.6 


17.1 



Table 6 Spearman correlations between number of selections and retention scores 





Corre 


ations 




r 


E. 


IL 


0.35 


0.05 


HK 


0.24 


0.13 
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since the number of looked up words was 12, the figures in Table_5 show that on average each 
word was looked up more than once. The log files showed that each individual word was look^ 
up between 1 and 3.2 times. It is clear that the Chinese learners looked up the words almost twice 
as often as the Israeli learners. However, as Table 6 shows, the relationship between the number of 
selections and retention is weak. App>arently people with a larger number of lookups do not 
necessarily remember more words (Chun & Plass, 1996). In the case of the Israeli learners, the 
correlations are low and in the case of HK learners, low and insignificant. 

DISCUSSION 

Incidental Vocabulary Acquisition and CALL Dictionary Information 

The use of a dictionary has been shown to have a positive effect on incidental vocabulary learmng 
(see Background section! . Our results Support this claim (see Tablej). Yet some studies show that L2 
readers often decide not to use the dictionary when meeting vmfamiliar words in a text (Hulstijn, 
1993; Bogaards, 1998). One of the reasons often reported by students is the time involved in 
flicking through the dictionary pages and the subsequent disruption of the flow of reading. An 
electronic dictionary may provide a good solution to this problem. The ease and speed of use may 
encourage the learner to look up unfamiliar words. This in turn will not only contribute to more 
fluent reading, but will also increase the chance of acquiring the looked up words. Since our study 
was not designed to. investigate the differences between paper and electronic dictionaries, the above 
advantage is still speculative. Yet we were encouraged to believe in it on the basis of a survey 
carried out among the HK learners al ter the experiment in which they evaluated the programme as a 
vocabulary learning tool: 97% of the HK subjects commented favourably on it and recommended 
further development A similarly enthusiastic response was found with the parhdpants in the 
studies by Roby (1991) and Lomicka (1998). If a pedagogical tool is popular with the students, the 
chances are it will also be beneficial for learning. A counter argument could be leveled at electromc 
dictionaries claiming that the ease of use will result in shallow processing of the looked up word 
and will therefore be detrimental to retention. Our results do not support this position. Any attempt 
to explain why this is so would be sjjeculative only. We would like to believe that the favourable 
attitude of the learner and the variety of lookup options resulted in careful attention to the lexical 
inlormation provided by the glosses. Further research comparing the programme with paper 
dictionaries could corroborate this speculation. 

How do our results compare with other studies where dictionaries were used for unknown words? 
Mondria (1993) found that alter looking up new words in a dictionary, learners remembered 15% 
of them on a post-test. Knight’s (1994) subjects recalled 20% of the tested words. According to 
Hulstijn Hollander, and Greidanus (1996), 25% of the looked up words were remembered when 
the word appeared in the text once. Chun and Plass (1996) report 25% accuracy in production and 
77% on recognition tests. The students in our study have outperlormed the subjects in the above 
studies. The Israeli group remembered’33.3% of the words, the HK group, 62%. There are 
several differences in the design of the studies. Our programme provided the learner with the 
choice of language of explanation, with the choice of information beyond mere word meaning, and 
with the choice of access to multiple items of inlormation. We would like to postulate that learners’ 
ability to select the type of information they consider most appropriate for the t^k and feel most 
comfortable with may well contribute to retaining more looked up words than in the other studies. 
Another contributing factor to our relatively good retention results may have been the combination 
of the different lookup possibilities with the specific task-text comprehension. In Mondria's 
( 1993) study, 14 sentences with 14 target words were presented to the learner. The students were 
asked to guess the words, then verify their meaning in a dictionary. The length ot text and number 
of target words are similar to our study, but the task is different. Mondria's task requires 
comprehending isolated sentences; our task requires comprehending a coherent text. 
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Comparing our study to Knight (1994) and Hulstijn et al. (1996), we can see that the global tasks 
assigned to students in the studies were similar read the text to perform a comprehension exercise 
or test. However, our study differs in two additional factors which may have affected the results: 
the quantity of text presented to the learner and pointing out the unknown words to be leaked up. 
Knight's students were tested after reading a text of 250 words with 14 target words which were 
not hi^lighted on the screen. Hulstijn et al used a text which was 1306 words long with 16 target 
words that were not presented in any way to the learner during the reading session. They had a 
dictionary and could look up any word they wanted. These differences, a shorter text in our study 
and highlighting the target words, meant that the words were more salient in our input than in the 
input of the other two studies. This input enhancement, which, in our study, can be attributed to 
the CALL technology, may partly be responsible for the better results than in other studies. 

We cannot, at this point, be certain why the Chinese learners did so much better than the Israeli 
learners. Based on our knowledge of the students in the experiment, we assume that it is because 
the Chinese learners are more diligent and more intrigued by the programme than the Israeli 
students. (As mentioned earlier the Israeli students spent about h^f of the time that was allocated 
them while the Chinese learners took up almost the entire 10 minutes.) As the programme was 
originally designed for Hong Kong University students (see Endnote 2) , those students may have 
felt that there was a certain "ownership" involved and that they should perhaps pay more attention 
to the information provided. A further factor could be that the Hong Kong study was conducted by 
the teacher who was also the researcher whereas in Israel; a research assistant collected the data. 
The Hong Kong students also tend to be competitive, so they attended to the lexical information 
more seriously: this greater attention may have resulted in better leaming.4 It is also possible that 
the Chinese learners are more word or vocabulary oriented, and that they have been trained to do 
more bottom-up processing. Conversely, it is possible that the Israeli learners approached the task 
from a top-down perspective in that their goal was comprehension of the text as a whole rather than 
knowing the meaning of each word in the text. 

Variability in Dictionary Lookup Patterns 

Even though a variety of dictionary information was available, most students opted for definitions, 
translations, or both. This reflects the findings ol Davis and Lyman-Hager (1997) and Lomicka 
(1998). Though it was originally anticipated that each option would provide a rich resource of 
research data, the total number of lookups for "extra information" and "root" formed only a small 
percentage (<5%) of the total selections. Therefore, these data do not merit discussion here. 

Table 3 shows clearly that different people have different lookup patterns. But it also reveals that the 
groups of learners in each country behave differently. The individual and group differences may 
have to do with individual learning styles, specific features of the learners’ mother tongue, or 
transfer of training. The Chinese LI students in our study tended to look up more information 
about the words than the Hebrew LI students. As mentioned above, this could reflect the study 
patterns of Chinese learners, who are often diligent It is also evident that Chinese learners prefer 
to look up English meanings of unfamiliar English words rather than LI translations. Israeli 
learners, on the other hand, have a noticeable preference for LI translation. A similar diversity of 
language lookup choice was found in Lomicla's (1998) study. Subjects who had access only to 
French and English glosses consulted the L2 definitions more frequently than LI. Those in the all 
glosses group, on the other hand, consulted LI more frequently than L2. Aust, Kelley, and Roby 
■ (1993) noted that bilingual dictionary users consulted 25% more definitions than monolingual 
dictionary users, but they add that their subjects' answers were written in LI (English) and this 
may have affected their bilingual lookup preference. The language of lookup did not make any 
significant difference on comprehension. They also point out that: 

The notion that greater improvements in vocabulary ability will result from monolingual 
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dictionary use seems reasonable because the cognitive tasks are more directly associated 
with understanding the foreign language than when the learner uses a bilingual dictionary 
and cycles from one language to another, (p. 71) 

In Davis and Lyman-Hager's (1997) study on multimedia glosses in French reading, however, the 
subjects tended to utilize almost exclusively word definitions provided in English (LI), ignoring 
the other forms of glosses available. 

In the present study, we postulate that these differences in preferred lookup language may be 
related to transfer of training. Most, if not all, of the Hong Kong subjects would have attended 
English medium secondary schools where their teachers may have encouraged them to consult 
monolingual dictionaries in their senior school classes. In Hong Kong universities, the medium of 
instruction is English and many, though certainly not all students, are accustomed to using 
monolingual dictionaries. No instruction, however, is given on the usage of dictionaries in their 
English enhancement courses. In Israeli universities, the medium of instraction is Hebrew and no 
specific preference is given to any type of dictionary. The available English-Hebrew dictionaries 
are quite good and most learners choose to use them for reading English texts. The preference of 
the Israeli learners cannot be attributed to inlerior dictionary use skills since they have b^n trained 
to use bilingual, monolingual, and bilingualized (English-English-Hebrew) dictionaries in high 
school. Yet most of them prefer a bilingual dictiona^ (Laufer & Kimmel, 1997). Lyman-Hager 
and Davis ( 1996) suggest that accessing word meanings in the native language is a key factor in 
comprehension. 

The records of lookup behaviour in log files showed that the Chinese learners check the 
phonological dimension of words, while the Israelis do not. Particularly, the Chinese learners 
check the pronunciation of the words which look as if they would not conform to standard English 
pronunciation rules, such as burgeoning. This difference between the groups may be related to the 
different types of orthography of the two languages (Chinese and Hebrew). It is arguable that the 
Israeli subjects, whose LI is written in an alphabetic orthography, are used to "sounding out" 
words, so that they are already good at decoding the sounds of English and may find the 
pronunciation of the words superfluous. Lomicka (1998) also noted that in her study of English LI 
subjects reading a French text, only 2 out of 12 subjects frequently consulted the pronunciation 
gloss and this "did not seem to directly affect comprehension" (p. 48). Chinese subjects, however, 
tend to focus on the written fonn of words, as with Chinese characters, and they appear to benefit 
from hearing the pronunciation of unfamiliar words as a means of retention (Hill, 1994, 1996). 

There is evidence to suggest that the average Hong Kong Chinese ESL student recognizes lexical 
items by orthographic (written form) rather than phonological (sound) principles (Hsia Chung & 
Wong, 1995). Hsia et al. points outs that young Cantonese speaking children in Hong Kong are 
not taught Cantonese sound analysis. Thus, visual memorization becomes the most mastered 
strategy. Students have reported visual memorization as being a dominant cognitive strategy in 
learning English words. However, this programme provides freely accessible pronunciation of 
unfamiliar words and so it is possible that the addition of auditor information has helped the 
Chinese learners to build referential connections between the written form and the meanings of the 
words. The bimodal (visual plus auditory) presentation of words may have enhanced the storage in 
short tenn memory, especially in proficient (tertiary level) L2 learners of English (Mayer & 

Simms, 1994). Another interesting speculation is that the Chinese preference for the pronunciation 
option could somehow be related to the fact that Chinese dictionaries are arranged according to the 
phonetic radical and so Chinese lookup words in a dictionary by sound. In the programme 
evaluation survey conducted al ter the experiment, students noted that it was useful to be able to 
hear the pronunciation of the words, although some commented that they had not checked the 
sound as they did not anticipate using these words orally. 
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Few learners (in Hong Kong and in Israel) paid attention to additional information provided in 
dictionaries, perhaps perceiving this as time consuming. And even fewer wer interested in the roots 
of words. In spite of the group characteristics discussed above, each group of learners includes 
different /ooitMp types, which demonstrates that different people, irrespective of what country they 
come from, have different dictionary consultation strategies. 

Lookup Patterns and Incidental Vocabulary Learning 

The interpretation of the results would have been simple if we had found a uniform relationship 
between lookup patterns and retention of looked up words. This is not the case in our study. The 
HK results show that LI lookups yielded the worst results. In the Israeli group, however, 
resorting to LI was the second best strategy, in absolute terms, and it was not significantly 
different from any of the other lookups. We cannot ascribe the overall better results of the Hong 
Kong learners to their preference to use English definitions of the new words. If the use of L2 
were a superior strategy, it should have produced better results than the use of Llwith the Israeli 
learners too, which was not the case. Besides, the use of LI is not necessarily an inferior learning 
strategy as it has long been established that L2-L1 pairs can be retained well (for a review, see 
Nation, 1982). Why then would the same lookup strategies work differently for each group? We 
do not know for sure (and the experiment was not designed to find this out). Yet we codd 
reasonably assume that since these groups of learners are accustomed to using dictionaries in 
different ways, as discussed in the previous section, good retention is the result of resorting to the 
lookup strategy learners feel most comfortable with. 

In spite of the differences between the two groups, our results show that the use of LI together 
with L2 leads to good retention (best scores in both groups, and ^ores which are not significantly 
different from the best scores in the Hong Kong group). If selection of both LI and L2 information 
means that the new word has been attended to more carefully (or noticed, elaborated, processed 
better, cf. Introduction) than the word which was looked up in one of the languages, then our 
results are in line with the claim that retention is determined by the way in which new words are 
processed, whether the learning is intentional or incidental. Ellis (1994) states that words may 
easily be forgotten after the first encounter, however, "explicit, deep, elalxjrative processing 
concerning semantic and conceptual/imaginal representations prevents this" (p. 52). The beneficial 
effect of L1+L2 lookup may lie in the richness of semantic encoding; it may lie in the prolonged 
attention that multiple items of information require; or it may lie in both. 

It is not clear at this stage why the number of selections did not correlate well with retention. 
According to Pimsleur's (1967) recommended memory schedule, which proceeds in intervals of a 
factor of two, (5 seconds after the first exposure, then 25 seconds, etc.), words that were looked 
up several times should have been remembered better than words that did not undergo immediate 
rehearsal. 

It is notable that the correlation was better and significant in the Israeli group. As stated earlier, 
most Israeli learners finished the tutorial within 5-6 minutes (out of the 10 minutes assigned for it), 
that is, were not very attentive. The significant (albeit not high) correlation in the Israeli group may 
suggest that learners who attended to die words more than once learned them better. The Chinese 
learners, who were attentive at first reading, may have done their main learning then (most of them 
spent closer to the ten minutes allocated). Apparently, when more time is spent on a word, 
additional selections do not add significantly. 

Teaching Implications 

Following the results of the study, we would like to make some recommendations for reinforcing 
vocabulary learning through reading in classrooms that have no access to CALL programmes. 
Teachers could assign the task of looking up specific words in paper dictionaries. These would be 
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words that teachers, on the basis of their experience, know are unfamiliar to their studente. 

Learners should also be encouraged to access different kinds of information found in their 
dictionary. They frequently check only the meaning, without looking any further for the examples 
of usage, or other lexical specifications. And yet it is accessing multiplicity of information that is 
likely to enhance retention. A further point worth considering is the use of bilingualised 
dictionaries, which contain the monolingual information about a word and its translation into the 
learner's mother tongue (see, for example, Oxford Student's Dictionary for Hebrew Speakers 
[Ruse, Reif, & Levy, 1978]), as such a dictionaiy caters for a variety of lookup preferences; for 
definitions and examples in English, translations into LI, or both. 

CONCLUSION 

The aim of the paper was twofold: to suggest a CALL methodology suitable for investigating 
vocabulary acquisition, and to investigate a possible relationship between lookup patterns and 
retention of looked up words. The novelty of the methodology lies in offering learners several 
options in selecting lexical inlormation about words. Researchwise, this means that incidental 
vocabulary learning can be observed under optimal lookup conditions since learners can select the 
look up strategy which may be most compatible with their learning style, whether it involves a 
particular lookup pattern or a combination of patterns. 

The results of our study suggest that such conditions, which cater to a variety of lookup 
preferences, may be more favourable for incidental vocabulary learning than dictionary use 
conditions reviewed in the introduction section. The variety of lookup preferences emerged with 
the comparison of learners in each country and, more so, with the comparison of learners in the 
two countries. We have not demonstrated conclusively that a particular lookup behaviour yields the 
best results. Nevertheless, the results suggest that multiplicity of lexical information tends to be 
associated with better retention. The number of times the word is looked up during a learning 
session bears almost no relation to its retention. We postulated, albeit cautiously, that what matters 
is greater attention during the lookup rather than the number of lookups. 

In addition to being a research tool. Words in your Ear and similar programmes written in the 
future can fulfill an important pedagogical function. First, the programme directs the learner's 
attention to unfamiliar words during reading, which in turn can contribute to incidental acquisition 
of these words. Second, it provides an on-line optimal dictionary. This dictionary is quick and 
easy to use and does not interrupt the flow of reading. It is also compatible with individual lookup 
preferences since it allows the user to select the kind of lexical information s/he feels most 
comfortable with. Furthermore, the various lookup options combine into multiple lexicd 
information for those learners interested in it or encouraged by the tether to use it Third, the 
programme will include special vocabulary exercises which will consist of a variety of input and 
output oriented tasks designed to reinforce the retention of the looked up words. Preliminary 
exercises (not included in our study) are already in the programme. Most of the Hong Kong 
learners who experimented with the text, dictionary and the preliminary exercises seemed to be 
involved with the programme and found it a great novelty. 

Finally, we would like to make some suggestions for further research. Our study could be 
replicated with larger samples in an attempt to find an unequivocal relationship between lookup 
pattern and retention. Moreover, unlearning of wrongly pierceived meanings of words can be 
studied by comparing incorrect meanings given in the pre-test with post-test meanings to see 
whether or not students had unlearned Ae original erroneous meanings and learned the correct 
meanings. An analysis of the log may shed some light on the conditions in which students can be 
helped to dispel wrong meanings and assimilate the correct ones. The reinlorcement exercises 
could be investigated to ascertain whether particular types of follow-up exercises can better aid 
retention. An experiment similar to ours could be conducted with paper dictionaries: bilingual, 
concise monolingual and detailed monolingual. Each student would be provided with the three and 
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would report on his/her look up behaviour for each word. And finally, our study tested retention 
immediately after the reading and look up task. Yet learning in real life requires retention of 
information long after the task performance. The relationship between lookup behaviour, 
reinforcement exercises and long term retention of vocabulary may well be the most important 
follow-up of our research. 

ENDNOTES 

1 Further consolidation of the knowledge of the word may involve associative learning such as 
semantic or imagery technique and rehearsal of the word in isolation, in an L2-L1 pair, in phrase or 
sentence context. Yet these activities are carried out by the learner with the specific intention of 
committing the word to memory and therefore belong to the realm of intentional learning. For an 
extensive discussion of incidental and intentional vocabulary learning, see Hulstijn (1999). 

2 This programme was developed as a part of the second author's Ph.D. thesis. Following the 
tutorial section described here, there is a selection of exercises which was not used for this 
particular experiment. 

3 One can argue that it is never possible to be sure whether learning is incidental, i.e. devoid of a 
decision to commit the information in question to memory. Some people may be carrying out a task 
other than, or additional to, the task assigned by the experimenter, such as memorizing words 
when they are asked to read or to look for grammatical patterns. Y et when experiments are carried 
out, we assume that the majority of subjects follow our instructions. 

4 The study was not designed to specifically compare the two groups of students. It mainly 
focused on within subject differences and intra-group differences as a function of different look up 
behaviour. Therefore, the comments explaining the differences between the HK and the Israeli 
students are based on what we observed during the experiment and what we know about our 
students. 
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